Learning What Is Relevant to the E ects of Actions for a Mobile Robot

We have developed a learning mechanism that allows robots to discover the conditional eeects of their actions. Based on sensorimotor experience, this mechanism permits a robot to explore its environment and observe eeects of its actions. These observations are used to learn a context operator diierence table, a structure that relates circumstances (context) and actions (operators) to eeects on the environment. From the context operator diierence table, one can extract a relatively small set of state variables, which simpli-es the problem of learning policies for complex activities. We demonstrate results with the Pioneer 1 mobile robot. 1 Introduction This paper describes a method by which a robot learns context-dependent eeects of its actions. The robot is a Pioneer 1, manufactured by Real World Interfaces, Inc., equipped with sonars and a gripper. By interacting with its environment, the robot learns a context operator diier-ence table (CODT) that relates primitive actions to changes in sensor readings. In general, the eeects of actions will depend on context. For instance, if the robot has run into an obstacle, then attempted forward movement will have little or no eeect on the values returned by forward-pointing sonars, whereas moving backward, away from the obstacle , will aaect these sonar readings. Each cell in a CODT speciies the eeects of a single action on readings from a single sensor in context. For example, when the robot is jammed against an obstacle, the left-wheel-stalled and right-wheel-stalled sensors return \true", and in this context , attempted forward movement will have no eeect on the robot's frontmost sonar, but attempted backward movement will be accompanied by an increase in the readings from that same sonar. Conditional knowledge of this sort is stored in decision trees, and the CODT contains one such tree learned for every action-sensor pair. Although CODTs represent the eeects of actions, they are not used to control the robot. Control policies are learned by another algorithm, described below. In fact, the